Horizontal Generalization

In conventional supervised learning, one searches for "vertical" patterns, connecting inputs directly to outputs. One can instead search for "horizontal" patterns, which go across the learning set, connecting one part of it with another. One way to do this is to pre-process the learning problem before feeding it to a conventional generalizer (like backpropagation). This pre-processing technique has been most fully worked out for the case where the input space is one-dimensional; in such a situation, the "pre-processing" which makes one search for horizontal patterns is simply the embedding process of non-linear time-series analysis. Although most natural for such onedimensional problems, one can investigate how well horizontal generalization works when the input space has more than one dimension. This paper presents two sets of experiments of the efficacy of horizontal generalization for such scenarios. The fIrst set of experiments takes place with a 24bit input space. For a number of different target functions, a learning set is constructed by sampling the target function at 301 separate points. A version of decision-directed learning using "horizontal" pattern-searching is then used to extrapolate from that learning set to the remaining 224 301 points in the input space. The average error rate was 8.6%. The second set of experiments was a variation of the noisy robot arm problem recently investigated by MacKay [23]. Here again decision-directed horizontal learning is used to do extrapolation. The rrns error rate was 3 times that of the noise.